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ABSTRACT 

Described is the Del Mod System, its inception, the 
fundings associated with it, and its objectives. A number of projects 
have already been completed and implemented. This monograph describes 
the evaluations of such projects and of the Del Mod systems-approach 
to science education. The area of evaluation and the method Jf data 
collection (with guidelines listed) are described. Problems of the 
System •s evaluation are noted. These include lack of financial 
resources, inadequacy of some designs, and public apprehension about 
research. References are cited. (EB) 
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BACKGROUND A Systems-Approach to Science Education was cre- 

ated in 1971 to serve as an integrative agency represent^ 
ing the Delaware Department of Public Instruction, the 
University of Delaware, Delaware Technical and Com- 
munity College, and Delaware State College. It has been 
jointly funded by the National Science Foundation, the 
State of Delaware, and the Du Pont Committee on I du- 
cational Aid, each of which has made a five-year com- 
mitment to the systems-approach. At the end of that 
time the various components are expected to assume 
appropriate responsibility for Del Mod functions. 

The Del Mod System is the first systems-approach 
to an educational problem ever funded by N.S.F. because 
there were several salient features in the Del Mod pro- 
posal of 1970 which appealed to them. One was the 
integrative nature of the systems-approach, which, if 
successful, will coordinate science-education activities of 
several highly autonomous institutions in Delaware. The 
potential for increased fiscal and intellectual efficiency 
was evident to N.S.F. and to many educators in the state. 

Another advantage of the Del Mod System will be its 
relative stability as a five-year project. N.S.F. tradi- 
tionally funded one-year projects and proven performers 
were frequently refunded. Occasionally, this method of 
funding was criticized because it prohibited development 
of long-range projects and long-range evaluation. And 
to some extent, it also prohibited the development of 
integrative projects. 

The primary objective of the Del Mod System is to 
increase the scientific literacy of children in Delaware 
schools through improved training of tlieir teachers. A 
number of projects have already been implemented or 
designed to meet that objective. The evaluations of such 
projects and of the Del Mod systems-approach to sci- 
ence-education are the subjects of this paper. 
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AREA Evaluation undertaken by Del Mod should accom- 

plish several specific goals: there must be a body of 
EVALUATION baseline data describing teachers, students, and schools 
in Delaware. These data will affect the evaluation of each 
project, each component institution, and the Del Mod 
System itself. 



Each project funded within the System should be 
evaluated. Since some project directors may have ques- 
tions about evaluation, the Research Director will assist 
in the development of both short range and longitudinal 
studies to measure the impact of the projects upon the 
students of teachers involved therein. 



Needs assessment should be conducted throughout the 
five year duration of the System. Agents arc being sent 
into the schools to assess needs, conduct research, and 
disseminate Del Mod findings. These agents are expected 
to develop sound professional recommendations about 
future Del Mod programs. 



Concomitant with these activities, a sociological eval- 
uation of the Del Mod organizational structure and its 
components should be conducted to ascertain lines of 
communication, degree of commitment, sources of power, 
and areas of weakness both within and external to the 
System. These evaluations will pinpoint potential weak- 
nesses in the communication flow and reveal constraints 
in the System itself. 

Finally, a panel of consultants should convene peri- 
odically to ask, "Is Del Mod doing what it set out to 
do?", "Is the payoff on each project worth the expense?'*, 
and, "Are the right variables being measured and are the 
analyses appropriate?" The Del Mod System snould re- 
direct its flow of resources and/or should redesign its 
plans, if necessary, to meet newly demonstrated needs. 
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COLLECTION The collection of baseline data was bf gun in August, 

1970, one year before the official beginnings of the Del 
DATA ^{qJ System. At that time a detailed study into the back- 
ground and preparation of Delaware secondary school 
science teachers was undertaken by the author. Three 
hundred and eighty teachers were surveyed, the college 
transcripts of each were analyzed, and the Test on 
Understanding Science (TOUS) was administered. All 
of the resultant data were computer analyzed and for- 
warded to N.S.F. as The Annual Report of the Del Mod 
Research Director^ in August, 1971. 

Student data were somewhat harder to collect. There 
was little time to plan a testing program in the 1970-1971 
school year, but the TOUS was administered to over 
1000 seniors from several high schools in the state. During 
the 1971-1972 school year fourth, eighth, and twelfth 
grade students have been given science achievement tests. 
The Department of Public Instruction in Delaware has 
tested the fourth and eighth graders using a test devel- 
oped for Delaware by Educational Testing Services, 
Princeton, New Jersey. This test, based on items from the 
STEP tests, will be analyzed by the Del Mod System 
during the month of June, 1972. Seniors in five randomly 
selected high schools were administered a modified ver- 
sion of the National Assessment Test. This test is cur- 
rently, being analyzed by the Del Mod System along 
lines similar to those in the report of the National As- 
sessment.^ 

Project evaluativ/n has been undertaken by the indi- 
vidual project directors along guidelines suggested by 
the Del Mod System. 

These guidelines are reproduced below: 

"Programs funded under the aegis of the Del Mod 
System should be quantitatively and qualitatively evalu- 
ated. The reasons for this are manifold: the taxpayer 
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deserves to know what his money is doing; private indus- 
try, NSF, and the State of Delaware will expect profes- 
sional appraisals of programs they have sponsored; and 
the inevitability of evaluation will tend to upgrade the 
various programs, 

"Evaluation should be consistent with the eleven 
guidelines from pages 6 and 7 of the Del Mod System 
Fiscal Procedures and Reporting Manual.' Thus, the 
proposal should iujlude: 



1. Objeetives of proposed project in behavioral terms 

2. A description of the target population as fitting the 
priorities of the Del Mod System. 

3. A brief summary of proposed activities high-lighting 
any new or special features 

4. A PERT Chart of activities 

5. An evaluation design 

6. A de.scription of leverage, including any additional 
sources of support and letters of commitment from 
cooperating sehools, if applicable 

7. A deseription of efficiency as part of assessing the 
input of the project on improving the quality of 
education without increasing eosts 

8. A description of systemization as it leads to the estab- 
lishment or enhancement of a mutually reinforcing 
relationship between the Del Mod System and the 
local schools 

9. A description of residual impact indicating how 
activity will be supported or phased into normal 
operation after support is terminated 

10. Any changes which might arise which might affect 
prcservice education, if applicable 

11. A budget relating to aetivities with all funding 
sources shown 



"The fifth guideline 'evaluation design' should be 
interpreted to mean statistical design for measuring the 
progress of the target population. Pre- and post-experi- 
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mental measures should be planned for every Del Mod 
program. Hard evidence will be an invaluable tool in 
explaining and accounting for success (or failure) of the 
various programs. The various funding agencies have 
indicated that they will not settle for less. 

"The proposal writer should suggest the best methods 
for evaluating his own program. To insure objectivity the 
research design should be reviewed carefully by the com- 
ponent coordinator and the Research Director of the 
Del Mod System. No proposal should be funded without 
an acceptable plan for statistical evaluation. 

"The final report for each funded project should be 
statistically supported so that any replication of the 
project will have an equal (or better) chance of suc- 
ceeding in another community.*** 

Two agents have been visiting each school in Dela- 
ware to assess the status of science education. Approxi- 
mately 150 science teachers will be videotaped by the end 
of the 1971-1972 school year, and the science facilities 
of each school will be evaluated by these agents. Fol- 
lowing the analysis of the videotapes, the tapes will be 
stored as part of the body of baseline data. Two re- 
searchers are currently employing the Flanders-Amidon 
Interaction Analysis Technique to evaluate the styles of 
teaching exhibited on the tapes. Their analyses will con- 
stitute doctoral dissertations for Temple University and 
the University of Texas, respectively. 

The sociological evaluation of the Del Mod System 
is deemed necessary if the systems-approach is to be 
adopted elsewhere. The political problems which must 
be overcome to insure the interrelationship of schools, 
colleges, and a department of public instruction are many 
and varied. After these are resolved, the intellectual 
problems of improving science-education can be dealt 
with. To date, a study of this nature has not been con- 
ducted. 
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EVALUATION 



External evaluations of the Del Mod System proposals 
arc conducted by the National Science Foundation as 
each annual supplemental budget request is submitted. 
This procedure docs not provide detailed review of Del 
Mod projects. Evaluation techniques should be assessed 
by a panel of consultants trained in educational research 
methods, and suggestions for more comprehensive studies 
could be planned in cooperation with the outside review 
team. 



PROBLEMS The Del Mod System was preceded by serious study 

OF and data collection which justified the development of 
.mArSikij ^ system and the support for the initial projects. Many 
of these recent studies are now considered outdated or 
inadequate in scope. 

In retrospect, much of the early rationale for the 
System and its projects was based upon intuition and 
expediency. For example, PurnelP demonstrated many 
of the weaknesses in science education in Delaware 
schools. This study agreed with surveys conducted by 
scholars elsewhere, i.e., that junior high schools and 
elementary schools needed the greatest amount of atten- 
tion to effect good science education. The conclusions of 
PurnelPs study were based upon survey data and sub- 
jective data; they were intuitively pleasing; and, they 
received a broad base of attention and support in Dela- 
ware. 

When the Del Mod System became viable in July, 
1971, new questions became evident, and some of them 
are difficult to answer with the initial research resources 
and limited funds. Testing student achievement, for 
example, is costly and is relevant to Del Mod needs and 
goals. The Department of Public Instruction, which has 
the responsibility for student testing in Delaware public 
schools, has the resources necessary for adequate evalua- 
tion and tested science achievement for grades 4 and 8 
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in the Spring of 1972. The testing plan needs to be ex- 
tended to other grade levels and the informntion fed 
back to the schools for planning purposes. 



Another problem has been the skepticism which 
many educators have of research and evaluation. This 
is true even where the advantages of research are obvious. 
In Delaware, the research which has been conducted in 
the past has not always been conducted responsibly, nor 
have the results of the research been adequately shared 
with, or explained to, the subjects of that research. The 
Del Mod System has engaged in a policy of (1) outlining 
each phase of the evaluation procedures with each 
project director and teacher involved in research, (2) 
confidential treatment of all data collected, and (3) 
sharing data only with the subjects of the research. This 
policy is time-consuming, but its advantages should 
compensate the System by increasing the willingness of 
participants to cooperate in the research endeavor. 



SUMMARY This monograph presents a review of the basic plan 

for conducting research and evaluation within the Del 
Mod System: Systems Approach to Science Education, 
which is currently in its first year of operation in the 
state of Delaware. The plan includes an outline of base- 
line data collected, evaluation designs for the various 
projects and components of the System, and future evalu- 
ations which must be considered if the systems-approach 
is to be replicated elsewhere. 

In addition, some of the problems of research within 
the System are discussed. These include lack of financial 
resources, inadequacy of some designs, public appre- 
hension about research, and some tactics which are being 
attempted to overcome these shortcomings. 

A systems-approach to education is an idea which is 
complex, yet simple. The pooling of resources to solve an 
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educational problem by colleges, universities, state de- 
partments of public instruction, and the schools them- 
selves is an idea worthy of consideration by others. It 
has afforded Delaware many economies, a large amount 
of inter-institutional communication, sharing of purpose, 
and educational innovations. It is too early to assess the 
intellectual benefits the approach has for the children 
in Delaware schools. This assessment cannot be com- 
pleted until the termination of the Del Mod project 
several years from now. 
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